Search for: All records

We explore the possibility of using natural language processing (NLP) and generative artificial intelligence (GAI) to streamline the process of thematic analysis (TA) for qualitative research. We followed traditional TA phases to demonstrate areas of alignment and discordance between (a) steps one might take with NLP and GAI and (b) traditional thematic analysis. Using a case study, we illustrate the application of this workflow to a real-world dataset. We start with processes involved in data analysis and translate those into analogous steps in a workflow that uses NLP and GAI. We then discuss the potential benefits and limitations of these NLP and GAI techniques, highlighting points of convergence and divergence with thematic analysis. Then, we highlight the importance of the central role of researchers during the process of NLP and GAI-assisted thematic analysis. Finally, we conclude with a discussion of the implications of this approach for qualitative research and suggestions for future work. Researchers who are interested in AI-assisted methods can benefit from the roadmap we provide in this study to understand the current landscape of NLP and GAI models for qualitative research. 

This paper examined the role of climate (e.g., interactions with others) in the skill development of engineering and physical science doctoral students. Skill development in graduate school often is connected to students’ primary funding mechanism, which enables students to interact with a research group or teaching team. Advisors also play a pivotal role in the engineering doctoral student experience; however, less is known about how positive mentoring influences specific skill development for engineering doctoral students. Analyzing data from the Graduate Student Funding Survey (n = 615), we focused analyses on three climate Factors (Advising climate; Faculty and staff climate; Peer climate) and specific skill development variables (research, teamwork and project management, peer training and mentoring, and communication). We found that advising climate was statistically significant for all four career-related skills, faculty and staff climate for peer training and mentoring skills only, and peer climate for both peer training and mentoring and communication skills. Our findings highlight the importance of climate from a variety of sources within engineering doctoral programs for the development of career-related skills. 

Feedback is a critical aspect of improvement. Unfortunately, when there is a lot of feedback from multiple sources, it can be difficult to distill the information into actionable insights. Consider student evaluations of teaching (SETs), which are important sources of feedback for educators. These evaluations can provide instructors with insights into what worked and did not during a semester. A collection of SETs can also be useful to administrators as signals for courses or entire programs. However, on a large scale as in high-enrollment courses or administrative records over several years, the number of SETs can render them difficult to analyze. In this paper, we discuss a novel method for analyzing SETs using natural language processing (NLP) and large language models (LLMs). We demonstrate the method by applying it to a corpus of 5000 SETs from a large public university. We show that the method can extract, embed, cluster, and summarize the SETs to identify the themes they contain. More generally, this work illustrates how to use NLP techniques and LLMs to generate a codebook for SETs. We conclude by discussing the implications of this method for analyzing SETs and other types of student writing in teaching and research settings. 

Abstract BackgroundThis paper begins with the premise that ethics and diversity, equity, and inclusion (DEI) overlap in engineering. Yet, the topics of ethics and DEI often inhabit different scholarly spaces in engineering education, thus creating a divide between these topics in engineering education research, teaching, and practice. PurposeWe investigate the research question, “How are ethics and DEI explicitly connected in peer‐reviewed literature in engineering education and closely related fields?” DesignWe used systematic review procedures to synthesize intersections between ethics and DEI in engineering education scholarly literature. We extracted literature from engineering and engineering education databases and used thematic analysis to identify ethics/DEI connections. ResultsWe identified three primary themes (each with three sub‐themes): (1) lenses that serve to connect ethics and DEI (social, justice‐oriented, professional), (2) roots that inform how ethics and DEI connect in engineering (individual demographics, disciplinary cultures, institutional cultures); and (3) engagement strategies for promoting ethics and DEI connections in engineering (affinity toward ethics/DEI content, understanding diverse stakeholders, working in diverse teams). ConclusionsThere is a critical mass of engineering education scholars explicitly exploring connections between ethics and DEI in engineering. Based on this review, potential benefits of integrating ethics and DEI in engineering include cultivating a socially just world and shifting engineering culture to be more inclusive and equitable, thus accounting for the needs and values of students and faculty from diverse backgrounds.